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EXECUTIVE SUMMARY 

This report is a sequel to Profile of the California Partnership Academies, 2009-2010. 1 The 
Profile gave an overview of California Partnerships Academies (CPAs), and included some 
comparisons between CPA students and all high school students in California. 

This report provides, for the first time, comparisons between students in each CPA and non¬ 
academy students at the same high school. Administrators and teachers can use this 
information to guide improvement of individual CPAs. 

Like the Profile, this report uses data on academy students from the October 2010 reports 
submitted by each CPA to the California Department of Education (CDE) for the 2009-10 school 
year. In addition, this report also uses information from the Standardized Testing and Reporting 
(STAR) program to compare the characteristics of academy and non-academy students, and to 
analyze scores from the California Standards Tests (CSTs). 

The original purpose of CPAs was to identify students who appeared to be at high risk of 
dropping out of school, and provide an educational experience that would motivate and enable 
them to obtain high school diplomas. Whether CPAs are achieving this purpose depends on the 
answer to three main questions: 

Are CPAs initially enrolling high-risk students? 

Do these students succeed in being promoted from one grade level to the next, and ultimately 
graduating? 

During their years in the academy, does the academic performance of high-risk students 
improve? 

This report provides partial answers to these questions, by comparing students in each CPA with 
non-academy students at the same high school. 

Comparing characteristics of academy and non-academy students in grade 10 

To determine whether CPAs are initially enrolling high-risk students, we compared the 
characteristics of academy and non-academy students in grade 10. CPAs are located in high 
schools where parents of 10 th graders have less education and lower income, and where more 
10 th graders are Latinos, English Language Learners, and had lower grade 9 test scores in English 
Language Arts. 

Within high schools that have CPAs, the academy 10 th graders also have parents with less 
education and income, compared with non-academy 10 th graders, and the academies enroll 
larger percentages of Latinos and African Americans. Academies enroll more students who have 
ever been classified as English Language Learners, but compared to non-academy students the 
academies have more 10 th graders who have been reclassified as proficient in English, and fewer 


1 Charles Dayton, Candace Hamilton Hester, and David Stern, Profile of the California Partnership 
Academies, 2009-2010. Career Academy Support Network, UC Berkeley, October 2011. Available at 
http://casn.berkeley.edu/downloads/CPA_Report_2009-10.pdf 
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who are still classified as English learners. Academies also enroll a lower percentage of special 
education students. As a result, the academies have fewer 10 th graders whose 9 th grade test 
scores were in the lowest quartile. When the comparison does not include special educations 
students, the academies show more students whose 9 th grade test scores were in the bottom 
three quartiles, and fewer in the top quartile. 

In addition to comparing these overall percentages, we also compared 10 th graders in each 
academy with non-academy 10 th graders at the same school. For each comparison, a standard 
statistical test was used to determine whether the difference was statistically significant 
(p<0.05), or whether the difference could likely be attributable to chance variation. Most of the 
differences were not statistically significant — meaning that in most CPAs the academy 10 th 
graders are similar to non-academy 10 th graders at the host school. 

But in many cases the academy 10 th graders are significantly different from their non-academy 
peers at the same school. For instance, 106 academies had significantly larger proportions of 
females in grade 10, compared to non-academy students at the same school, while 71 
academies had significantly larger proportions of males. This reflects the fact that some 
industry sectors, e.g. health, enroll more female students, while others, e.g. building trades, 
enroll more males. 

Consistent with the intent of the law, the number of CPAs that over-represent low-income 10 th 
graders is larger than the number that under-represent this group. More CPAs also over¬ 
represent 10 th graders whose parents have low education, and more academies over-represent 
Latinos. On the other hand, more CPAs under-represent 10 th graders in Special Education, and, 
even when Special Education students are not included in the analysis, academy 10 th graders 
more often had higher average test scores in grade 9 compared to non-academy students at the 
same school. In CPAs where initial enrollment patterns do not seem consistent with the intent 
of the law, administrators and teachers can use these results to examine and improve local 
practices. 

Comparing characteristics of academy and non-academy students in grade 11 

To get an indication of whether certain kinds of students are more likely to continue from grade 
10 to grade 11, we compared academy and non-academy 11 th graders. This is not a truly 
longitudinal analysis, because the 11 th graders analyzed here are a different cohort of students 
than the 10 th graders. In fact, some of the academies with 10 th graders were new in 2009-10 
and did not yet enroll 11 th graders. 

Comparisons with all academies combined showed differences between academy and non¬ 
academy students in grade 11 that are similar to the differences in grade 10. It remains true 
that academies are located in schools where parents of 11 th graders have lower education and 
income, and more 11 th graders are Latino. 

Within these schools that have CPAs, the academy 11 th graders have parents with lower 
education and income, compared with non-academy 11 th graders, and larger proportions of the 
academy students are Latino and Filipino. These differences between academy and non- 



COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


academy students in schools that have CPAs are all somewhat bigger in grade 11 than in grade 
10 . 

Comparing grade 11 students in each CPA with non-academy 11 th graders at the same school 
also shows similar patterns in grade 11 as in grade 10. In most CPAs, the differences between 
academy and non-academy students at the same school were not statistically significant. Again 
the differences that are significant more often show academies enrolling more 11 th graders 
whose parents have low education and income, and more students who are Latino or Filipino. 
On each of these characteristics, the preponderance is greater in grade 11 than in grade 10. 

Generally, comparisons with all CPAs combined, or with each CPA compared to its host school, 
showed CPAs were more likely to enroll challenging or under-served populations of students in 
grade 11 than in grade 10. The only exception is that African American students were more 
often under-represented in CPAs in grade 11 than in grade 10. Again, it is important to recall 
that we are comparing two different cohorts of students, one in grade 10 and the other in grade 
11, so this is not a truly longitudinal analysis. 

Comparing performance of academy and non-academy students 

We compared performance of academy and non-academy students on the 2010 California 
Standards Tests. A separate regression analysis was run for each school and grade level. 
Predictors of 2010 scores included test scores in the same subject for 2009 and 2008, individual 
student characteristics, and whether a student was in an academy (or which academy, if the 
school had more than one). The coefficient on the academy variable indicates whether 
academy students' average score was better, worse, or not significantly different from non¬ 
academy students' score, taking all the other predictors into account. 

Most of the academy coefficients were not statistically significant. This result is consistent with 
previous studies. Again there is important variation. Some academies show significant positive 
coefficients, and some are significantly negative. 

Results for a single year are not conclusive. But compiling this information year after year, and 
including other outcomes in addition to test scores — such as attendance, credits earned, 
grades, behavior, and promotion to the next grade level or graduation from grade 12 — can be 
very helpful in guiding the continuous improvement of academies. 

Information to guide improvement of individual academies 

Academies vary a great deal in who enrolls, who stays, and whether performance improves. As 
the number of academies and Linked Learning pathways expands, it becomes increasingly 
important to monitor each program on an ongoing basis. We illustrate how the information in 
this report can be summarized in an accessible way for an individual academy. Districts and 
schools can use this kind of profile to help guide continuous improvement. 
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INTRODUCTION 

This report is a sequel to Profile of the California Partnership Academies, 2009-2010 , 2 The 
Profile gave an overview of California Partnerships Academies (CPAs), and included some 
comparisons between CPA students and all high school students in California. 

This report provides, for the first time, comparisons between students in each CPA and non¬ 
academy students at the same high school. Administrators and teachers can use this 
information to guide improvement of individual CPAs. 

Like the Profile, this report uses data on academy students from the October 2010 reports 
submitted by each CPA to the California Department of Education (CDE) for the 2009-10 school 
year. In addition, this report also uses information from the Standardized Testing and Reporting 
(STAR) program to compare the characteristics of academy and non-academy students, and to 
analyze scores from the California Standards Tests (CSTs). 

For readers who do not have the Profile report, the remainder of this Introduction reproduces 
paragraphs from that report, providing background on the California Partnership Academies. 

In 1984, the California State Legislature passed Assembly Bill 3104, launching the California 
Partnership Academies (CPAs). The authorization was renewed by Senate Bill 605 in 1987, 

Senate Bill 44 in 1993, and Senate Bill 1354 in 2010. Additional funding came as a result of 
Senate Bill 70 in 2006 for career-technical pathways, and Assembly Bill 519 in 2009 for "green" 
academies. With each round of additional funding, CDE has issued a request for proposals, and 
has awarded grants after a review of competing applications. In 2009-10 there were almost 500 
CPAs, enrolling close to 50,000 students in high schools located in 36 different counties. 

CPAs have served as a model for high school reform within and beyond California. Many of the 
districts that received federal Small Learning Community (SLC) grants employed "career 
academies," often based on California's CPAs in some or all of their SLCs. More recently, in 
school districts receiving grants from the James Irvine Foundation under its Linked Learning 
initiative, most of the Linked Learning pathways have been career academies. It is estimated 
that roughly 700 career academies exist in California and 7,000 in the nation, with both numbers 
continuing to grow. 

More than two decades of evaluations, beginning in the 1980s, have consistently found that 
career academies, including CPAs, produce positive outcomes for high school students (for a 
summary of the research, see Stern, Dayton, & Raby 2010). While not all comparisons between 
academy and non-academy students have been statistically significant, virtually all of the 
statistically significant differences have indicated better results for academy students. 

Several studies, comparing academy students with similar students in the same high schools, 
found that academy students over the course of their high school years had significantly 


2 Charles Dayton, Candace Flamilton Hester, and David Stern, Profile of the California Partnership 
Academies, 2009-2010. Career Academy Support Network, UC Berkeley, October 2011. Available at 
http://casn.berkeley.edu/downloads/CPA_Report_2009-10.pdf 
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improved attendance, earned better grades, completed more course credits, and were less 
likely to leave high school. A major study by MDRC, using a strict random-assignment design, 
found that academy students not only improved their performance while in high school but also 
had significantly greater earnings eight years after high school (Kemple 2008). Other studies also 
have found that career academies do, in fact, effectively prepare students for careers. 

At the same time, studies have found that career academy students also perform well in 
postsecondary education. The MDRC study found high rates of postsecondary educational 
attainment among former career academy students. Another notable study, by Maxwell (2001), 
found that career academy graduates who went on to one of the large California public 
university campuses were more likely to complete bachelor's degrees, and less likely to need 
remedial classes along the way, than other graduates from the same urban school district. The 
strong and consistent track record of career academies is one reason for their continued growth. 

CPAs combine a number of high school reform features generally considered to be effective. 
They: 

• Group 10th- through 12th-grade students into several related classes each year. 

• Organize cross-curricular teacher teams, both academic and career-technical. 

• Frame the academic classes within a broadly defined career theme, while in most cases 
still enabling students to complete the "a-g" course sequence required for UC and CSU. 

• Show students connections between their academic subjects and this career theme. 

• Show students connections between their coursework and activities outside the high 
school. 

• Incorporate employer and community support through advisory groups, speakers, field 
trips, job shadowing, mentors, and internships. 

Details of this model are available at the CASN website (http://casn.berkeley.edu), as well as the 
CDE website (http://www.cde.ca.gov/ci/gs/hs/cpagen.asp). 

Among other features of the law that governs CPAs, at least 50% of the students in each 
incoming class of CPA sophomores must meet three of the following six "at-risk" criteria 
(defined more specifically in the law): having a poor attendance record, being significantly 
behind in credits, demonstrating low motivation for the regular school program, being 
economically disadvantaged, having low state test scores, and having a low grade point average. 

CPAs also are required to match their State grant with either funding or in-kind support from 
both the receiving district and the academy's supporting employers, thereby substantially 
increasing the value of the State grant. Furthermore, CPA funding is performance-based, 
dependent on how many students meet specified targets for attendance, credits and graduation. 
CPAs are required to submit yearly performance data for each enrolled student, and they 
receive funding only for students who have met or exceeded the performance targets, up to a 
maximum or "cap". The maximum annual State grant for a CPA in 2009-10 was either $69,120 
or $81,000, depending on which State law authorized the funding. 
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THREE ESSENTIAL QUESTIONS 


The original purpose of CPAs was to identify students who appeared to be at high risk of 
dropping out of school, and provide an educational experience that would motivate and enable 
them to obtain high school diplomas. Whether CPAs are achieving this purpose depends on the 
answer to three main questions: Are CPAs initially enrolling high-risk students? Do these 
students succeed in being promoted from one grade level to the next, and ultimately 
graduating? During their years in the academy, does the academic performance of high-risk 
students improve? This report provides partial answers to these questions, by comparing 
students in each CPA with non-academy students at the same high school. 

1. Who enrolls? As noted in the Introduction, at least half the students entering each CPA in 
grade 10 must meet at least three of six specific criteria indicating higher risk of dropping out. 
CDE monitors whether individual CPAs meet this requirement. However, CPAs tend to be 
located in high schools where many or most students would satisfy the at-risk criteria. The law 
does not require CPAs to enroll students who are more at-risk than other students at the same 
school, and CDE has not monitored that. Given the original intent of the law, our analysis seeks 
to determine whether each CPA is enrolling sophomores whose demographic characteristics 
indicate that they face at least as much challenge as other sophomores at the same school. 

2. Who stays? CPAs are intended to enroll at-risk students, and then enable them to graduate. 
To know whether at-risk sophomores who enter an academy ultimately have higher graduation 
rates than similar sophomores at the same school, it would be necessary to follow a cohort of 
grade 10 students through grades 11 and 12. In the future we hope to do that analysis with 
actual longitudinal data. In this report we simply compare juniors in each CPA with other juniors 
at the same school. This is not a true longitudinal analysis, but it does reveal whether an 
academy is continuing to enroll grade 11 students whose demographic characteristics indicate 
higher risk of dropping out, compared to other juniors at the same school. It is not possible to 
do this comparison for grade 12 because the STAR assessment does not include grade 12. 

3. Does performance improve? In addition to whether students are promoted from one grade 
to the next and ultimately graduate, it is also important to know whether their academic 
performance improves. In the future we hope to compare progress of academy and non¬ 
academy students on a range of outcomes, including attendance, credits, and grades, as well as 
test scores. In this report we are limited to test scores. We analyze whether academy students 
in grades 10 and 11 perform better or worse than non-academy students at the same high 
school on the California Standards Tests in English Language Arts and in mathematics. The 
analysis uses multiple regression to control for prior test scores and other student 
characteristics. However, because of possible unmeasured differences between academy and 
non-academy students, this analysis may not measure the true causal impact of academies on 
test scores. 

We also provide an example of how all this information can be summarized for an individual 
academy or pathway, as a guide to continuous improvement. As the number of academies and 
pathways expands, this kind of information tool will become increasingly important. 
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WHO ENROLLS? 


Comparing Characteristics of Grade 10 Academy and Non-Academy Students 

We report two kinds of comparison: first with all academies combined, then for individual 
academies. 

Comparing 10 th grade academy and non-academy students in all academies combined 

Here we compare the characteristics of all CPA students in grade 10 (n=19,051) with all grade 10 
non-academy students in the state (n=480,708), and with the subset of grade 10 non-academy 
students who were in high schools that had CPAs (n=119,774). 

High schools with CPAs enroll students who have lower income and lower achievement, 
compared to the state as a whole. For example, Figure 1 shows that 48.5 percent of all non¬ 
academy 10 th graders in the state were eligible for the National School Lunch Program. 3 In 
schools that had CPAs, 58.4 percent of non-academy 10 th graders were eligible, and the 
eligibility rate for academy students was 62.9 percent. Clearly, high schools with CPAs enrolled 
relatively more low-income students, and the academies also had larger proportions of low- 
income students than the rest of those schools. 

Figure 1: Percent of grade 10 academy and non-academy students eligible for subsidized lunch 
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Academy students Non-academy students in all high Non academy students in high schools 
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Similarly, Figure 2 shows non-academy students in high schools with CPAs had less educated 
parents than non-academy students in all high schools. Within high schools that had CPAs, 
academy students also had less educated parents than non-academy students. 


3 In 2009-10, students were eligible for free lunches if their reported family income was no more than 1.3 
times the Federal poverty level. If family income was between 1.3 and 1.85 times the poverty level, 
students were eligible for subsidized lunch. 
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Figure 2: Percent of grade 10 academy and non-academy students by parents' education 
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There is little difference between the percent of females among non-academy students in 
schools with CPAs compared to non-academy students in all high schools. But CPAs enrolled a 
larger percentage of females in grade 10, as Figure 3 shows. 

Figure 3: Percent of females among grade 10 academy and non-academy students 
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In Figure 4 we see that CPAs had large proportions of Latino, Filipino, and African American 
students compared to non-academy students in the same schools, and even more so compared 
to non-academy students in all schools. Conversely, the proportion of white students was 
lowest in the CPAs. 


Figure 4: Percent of grade 10 academy and non-academy students by race or ethnicity 
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Economic disadvantage and low achievement are among the required criteria for selecting CPA 
students in grade 10, but English language fluency is not. Figure 5 shows that 57.5 percent of 
academy 10 th graders did not have English as their first language, compared to 53.6 percent of 
non-academy students in the same schools, and 42.9 percent of non-academy students in all 
schools. However, compared to non-academy students in the same schools, a larger proportion 
of CPA students had been reclassified as Fluent English Proficient, and a smaller proportion were 
still classified as English Language Learners. 
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Being in Special Education (with an Individual Education Plan) also is not one of the criteria for 
defining the "at-risk" students CPAs are required to enroll in grade 10. Figure 6 shows that the 
proportion of Special Education students in CPAs is lower than among non-academy students. 

Figure 5: Percent of grade 10 academy and non-academy students ever classified as English 
Language Learners 
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Figure 6: Percent of grade 10 academy and non-academy students in special education 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Low test scores in grade 9 are one of the criteria for identifying the "at-risk" students CPAs are 
required to admit in grade 10. Compared to non-academy students, more academy 10 th graders 
had grade 9 English Language Arts (ELA) scores in the middle two quartiles, according to Figure 
7. 4 CPAs enrolled fewer 10 th graders whose grade 9 ELA scores were in the bottom quartile, but 
this is partly attributable to the smaller proportion of Special Education students in CPAs. Figure 
8 shows that, when Special Education students are not included in the comparison, the 
proportion of students in the lowest quartile was larger in CPAs than for all non-academy 
students, though it was still smaller compared to non-academy students in schools that had 
academies. 


Figure 7: Percent of grade 10 academy and non-academy students by English Language Arts 
test score quartile in grade 9, all students 
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4 The boundary scores defining quartiles were determined from the distribution of grade 9 ELA scale 
scores for all academy and non-academy 10 th graders in all schools. 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 8: Percent of grade 10 academy and non-academy students by English Language Arts 
test score quartile in grade 9, without Special Education students 
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Comparing 10 th graders in each academy with non-academy 10 th graders at the same school 

The preceding figures showed results when grade 10 students from all CPAs were combined and 
compared with non-academy 10 th graders. The following set of figures show results of 
comparing grade 10 students in each CPA with non-academy 10 th graders at the same school. 
This kind of comparison can be useful for guiding improvement of individual CPAs. 

For instance, the combined comparison in Figure 1 showed that, overall, 10 th graders in CPAs 
were more likely to be eligible for subsidized lunch, compared to non-academy 10 th graders. 
Accordingly, Figure 9 shows there were 43 individual CPAs where the proportion of 10 th graders 
eligible for subsidized lunch was significantly greater than among non-academy 10 th graders at 
the same school. However, Figure 9 shows that 35 individual CPAs were exceptions to the 
overall pattern. In these 35 CPAs the proportion of 10 th graders eligible for subsidized lunch was 
significantly less than among non-academy 10 th graders in the same high school. For the large 
majority — 311 CPAs — the difference in subsidized lunch eligibility between academy and non¬ 
academy 10 th graders was not statistically significant. 5 


5 Tests of statistical significance were conducted separately for each academy. In comparing student 
characteristics that were measured as proportions, as in Figure 9, a chi-square test was used to gauge 
statistical significance. When the student characteristic was measured as a mean, as in Figure 17, a t-test 
was used. The criterion for statistical significance was p<.05. In each of these charts, therefore, it would 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 9: Number of academies in which the proportion of grade 10 students who were 
eligible for subsidized lunch was higher, not significantly different, or lower than among grade 
10 non-academy students at the same school 
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Too few students or no variance = ll 6 


Figure 10 displays a similar pattern with respect to parents' educational attainment. In the great 
majority of CPAs, parents' education was not significantly different for academy and non¬ 
academy students. In 24 CPAs (5 percent of the 383 academies that could be analyzed in this 
Figure), parents' education was significantly lower for academy than non-academy 10 th graders. 
This is consistent with the mission of CPAs to serve at-risk youth. However, in 20 CPAs parents' 
education was significantly higher for academy than for non-academy students. Administrators 
and teachers in these 20 CPAs might want to look into why these academies are enrolling more 
students of highly educated parents and, if appropriate, make some changes in recruitment 
procedures. 


be expected that 5 percent of the academies would show statistically significant differences just by 
chance. 

6 "Too few students" means there were fewer than 30 students at this school and grade level for whom 
information on this characteristic was provided. "No variance" means that all students were reported to 
be the same with respect to this characteristic. Tests of statistical significance are not appropriate under 
either of these conditions. 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 10: Number of academies in which grade 10 students had parents with higher, not 
significantly different, or lower levels of education compared to grade 10 non-academy 
students at the same school 
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Gender is not one of the "at-risk" characteristics specified in the law. What is interesting about 
Figure 11 is the relatively large number of CPAs that enrolled either significantly more females 
(106 academies) or significantly fewer (71 academies), compared to non-academy 10 th graders 
at the same high school. This reflects the fact, as documented in the 2011 Profile of California 
Partnership Academies, that some career fields are still predominantly female (e.g. fashion) or 
predominantly male (e.g. building and construction). In academies where this gender imbalance 
is occurring, administrators and teachers may want to consider how they can work toward a 
more balanced distribution, in accordance with Federal Title IX guidelines. 

Figure 11: Number of academies in which the proportion of grade 10 students who were 
female was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figures 12 through 16 display differences in enrollment of students from the largest racial and 
ethnic groups. Consistent with the overall percentages in Figure 4, the numbers of CPAs with 
significantly larger shares of Latino and Filipino students exceed the numbers of CPAs with 
significantly smaller shares of Latino and Filipino students. And the number of CPAs with 
significantly smaller proportions of Asian students exceeds the number with larger proportions 
of Asian students — also consistent with Figure 4. Flowever, a considerable number of 
academies show a different pattern than the overall percentages in Figure 4. For instance, 

Figure 14 shows 41 CPAs enrolled significantly lower proportions of Latino students, compared 
to non-academy students at the same school. This illustrates how comparing academy and non¬ 
academy students one school at a time can give different, and potentially more useful, 
information than simply comparing the overall numbers for the state. 

In two instances, comparing individual academies produces the opposite result from comparing 
all academies combined. The overall comparisons in Figure 4 showed CPAs generally enrolling a 
larger fraction of African American students, and a smaller fraction of white students. However, 
Figure 15 shows there were 18 CPAs where the proportion of African American 10 th graders was 
significantly less than among non-academy 10 th graders at the same school. Only 8 CPAs had 
significantly greater proportions of African American students. Similarly, Figure 16 shows the 
number of CPAs with significantly larger proportions of white students exceeded the number 
with significantly smaller proportions of whites — in contrast to the overall comparison in Figure 
4. In both cases, the seeming inconsistency could easily be explained by a somewhat larger 
fraction of African Americans, and a somewhat smaller fraction of whites, in the large group of 
CPAs where the difference between academy and non-academy students was not statistically 
significant. The point is that patterns within individual schools can be very different from the 
overall pattern. 


Figure 12: Number of academies in which the proportion of grade 10 students who were 
Asian was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 13: Number of academies in which the proportion of grade 10 students who were 
Filipino was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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Figure 14: Number of academies in which the proportion of grade 10 students who were 
Latino was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 15: Number of academies in which the proportion of grade 10 students who were 
African American was higher, not significantly different, or lower than among grade 10 non¬ 
academy students at the same school 
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Figure 16: Number of academies in which the proportion of grade 10 students who were 
white was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


With respect to grade 9 test scores in English Language Arts, Figure 17 shows that more CPAs 
had incoming 10 th graders with significantly higher average grade 9 test scores, compared to 
non-academy 10 th graders in the same school, even when Special Education students are not 
included in the analysis. Figure 18 shows a similar but less pronounced result for grade 9 test 
scores in math. As with the other student characteristics, in most CPAs the test score difference 
between academy and non-academy students was not statistically significant. But the numbers 
of CPAs with significant differences in Figures 17 and 18 is unusually large — much more than 
the 5 percent of the total that would be expected by chance. This suggests that some kind of 
polarization seems to be happening in terms of test scores. A large number of CPAs are 
enrolling 10 th graders who had either significantly higher or lower grade 9 test scores compared 
to non-academy students at the same school. Since CPAs are intended to enroll students with 
low prior test scores, it is of some concern that a considerable number of CPAs are doing just 
the opposite. Again, this is an issue that could be taken up by local administrators and teachers. 

Figure 17: Number of academies in which grade 10 students had higher, not significantly 
different, or lower mean test scores in English Language Arts in grade 9, compared to grade 10 
non-academy students at the same school, without special education students 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 18: Number of academies in which grade 10 students had higher, not significantly 
different, or lower mean test scores in mathematics in grade 9, compared to grade 10 non¬ 
academy students at the same school, without special education students 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


WHO STAYS? 

Comparing Characteristics of Grade 11 Academy and Non-Academy Students 

To get an indication of whether certain kinds of students are more likely to continue from grade 
10 to grade 11, we compared academy and non-academy 11 th graders. This is not a truly 
longitudinal analysis, because the 11 th graders analyzed here are a different cohort of students 
than the 10 th graders. They are not even from exactly the same set of CPAs, because some of 
the academies with 10 th graders were new in 2009-10 and did not yet enroll 11 th graders. 

As with the analysis of grade 10 students, we report two kinds of comparison: first with all 
academies combined, then for individual academies. 


Comparing 11 th grade academy and non-academy students in all academies combined 

Here we compare the characteristics of all CPA students in grade 11 (n=14,292) with all grade 11 
non-academy students in the state (n=463,768), and with the subset of grade 11 non-academy 
students who were in high schools that had CPAs (n=lll,648). 

Figures 19-26 show that the differences between academy and non-academy students in grade 
11 are similar to the differences in Figures 1-8 for grade 10. CPAs are located in schools where 
parents of 11 th graders have lower education and income, and more 11 th graders are Latino. 

Even within these schools that have CPAs, the academy 11 th graders have parents with lower 
education and income, compared with non-academy 11 th graders, and larger proportions of the 
academy students are Latino and Filipino. These differences between academy and non¬ 
academy students in schools that have CPAs are all somewhat bigger in grade 11 than in grade 
10. Although this is not a true longitudinal comparison, it suggests that academy students 
whose parents have low income or education, or who are Latino or Filipino, have at least as 
good a chance of moving from grade 10 to grade 11 as non-academy students do. 

Within schools that have CPAs, academy 11 th graders include more females, compared to non¬ 
academy 11 th graders. Academy students are also more likely ever to have been classified as 
English Language Learners but, in grade 11 as in grade 10, the academies over-represent 
students who have been reclassified as Fluent English Proficient, and under-represent students 
who remain classified as English Language Learners. Special Education students also are still 
under-represented in academies at grade 11, but to a smaller degree than at grade 10. The 
pattern of grade 9 test scores in English Language Arts among 11 th graders is similar to the 
pattern among 10 th graders: academies over-represent students in the middle two quartiles, and 
have lower proportions of students who scored either in the top of bottom quartile in grade 9. 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 19: Percent of grade 11 academy and non-academy students eligible for subsidized 
lunch 
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Figure 20: Percent of grade 11 academy and non-academy students 
by parents' education 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 21: Percent of females among grade 11 academy and non-academy students 
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Figure 22: Percent of grade 11 academy and non-academy students by race or ethnicity 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 23: Percent of grade 11 academy and non-academy students ever classified as English 
Language Learners 
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Figure 24: Percent of grade 11 academy and non-academy students in special education 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 25: Percent of grade 11 academy and non-academy students by English Language Arts 
test score quartile in grade 9, all students 
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Figure 26: Percent of grade 11 academy and non-academy students by English Language Arts 
test score quartile in grade 9, without special education students 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Comparing 11 th graders in each academy with non-academy 11 th graders at the same school 

The preceding figures showed results when grade 11 students from all CPAs were combined and 
compared with non-academy 11 th graders. The following set of figures show results of 
comparing grade 11 students in each CPA with non-academy 11 th graders at the same school. 

Generally, Figures 27-36 show similar patterns in grade 11 as Figures 9-18 depicted in grade 10. 
In most CPAs, the differences between academy and non-academy students at the same school 
were not statistically significant. Again the differences that are significant more often show 
academies enrolling more 11 th graders whose parents have low education and income, and 
more students who are Latino or Filipino. On each of these characteristics, the preponderance 
is greater in grade 11 than in grade 10. 

For instance, at grade 11, Figure 27 shows academies that had significantly more low-income 
students (as indicated by eligibility for free or subsidized lunch), outnumbered academies that 
had significantly fewer low-income students, by 39 to 20. In grade 10, Figure 9 showed the 
corresponding difference was 43 to 35. This indicates that CPAs were fulfilling the intent of the 
law to a somewhat greater degree in grade 11 as in grade 10. And in the 20 academies that 
significantly under-represented low-income students in grade 11, administrators and teachers 
could seek to find out why, and take appropriate corrective action. 

Special Education students were still significantly under-represented in CPAs in grade 11 more 
often than they were over-represented (by a margin of 47 to 8, as shown in Appendix Table 4), 
but this was less than the margin of 70 to 10 in grade 10 (Appendix Table 2). And, when Special 
Education students are not included in the analysis, Figures 35 and 36 show a greater tendency 
in grade 11 than in grade 10 for CPAs to over-represent students who had low test scores in 
grade 9, before they entered the academy. 

Generally, these comparisons show CPAs were more likely to enroll challenging or under-served 
populations of students in grade 11 than in grade 10. The only exception is that African 
American students were more often under-represented in CPAs in grade 11 than in grade 10. 
Again, it is important to recall that we are comparing two different cohorts of students, one in 
grade 10 and the other in grade 11, so this is not a truly longitudinal analysis. 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 27: Number of academies in which the proportion of grade 11 students who were 
eligible for subsidized lunch was higher, not significantly different, or lower than among grade 
10 non-academy students at the same school 



265 










39 

20 

Academy had more or higher No difference or mixed Academy had less or lower 


Too few students or no variance=7 


Figure 28: Number of academies in which grade 11 students had parents with higher, not 
significantly different, or lower levels of education compared to grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 29: Number of academies in which the proportion of grade 11 students who were 
female was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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Figure 30: Number of academies in which the proportion of grade 11 students who were 
Asian was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 31: Number of academies in which the proportion of grade 11 students who were 
Filipino was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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Figure 32: Number of academies in which the proportion of grade 11 students who were 
Latino was higher, not significantly different, or lower than among grade 10 non-academy 
students at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 33: Number of academies in which the proportion of grade 11 students who were 
African American was higher, not significantly different, or lower than among grade 10 non¬ 
academy students at the same school 
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Figure 34: Number of academies in which the proportion of grade 11 students who were white 
was higher, not significantly different, or lower than among grade 10 non-academy students 
at the same school 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 35: Number of academies in which grade 11 students had higher, not significantly 
different, or lower mean test scores in English Language Arts in grade 9, compared to grade 10 
non-academy students at the same school, without special education students 
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Figure 36: Number of academies in which grade 11 students had higher, not significantly 
different, or lower mean test scores in mathematics in grade 9, compared to grade 10 non¬ 
academy students at the same school, without special education students 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


DOES PERFORMANCE IMPROVE? 

Comparing Performance of Academy and Non-Academy Students 

The question whether students' performance improves also can and should be answered for 
each academy. Table 1 provides an example of how this can be done, using 2010 scale scores on 
California Standards Tests. A separate regression analysis was run for each school and grade 
level. Predictors of 2010 scores included test scores in the same subject for 2009 and 2008, the 
individual student characteristics listed in Appendix Tables A-l and A-3, and whether a student 
was in an academy (or which academy, if the school had more than one). The coefficient on the 
academy variable indicates whether academy students' average score was higher, lower, or not 
significantly different from non-academy students' score, taking all the other predictors into 
account. 

It is important to emphasize that these coefficients may not give an accurate indication of 
whether or how much academies cause students' test scores to change. The reason is that 
academy students may differ from non-academy students in ways that are not measured in the 
data. For instance, it is possible that students in some academies have higher levels of 
motivation than non-academy students. In other academies, the academy students may have 
lower levels of motivation than non-academy students. If motivation affects test scores, and if 
the differences in motivation were present before students enrolled in academies, these 
unmeasured pre-existing differences in motivation would bias the academy coefficient in the 
regressions. The regression coefficient then would overstate or understate the true causal 
impact of the academy on test scores. 

Table 1 shows most of the academy coefficients were not statistically significant. This is 
consistent with previous studies. 

Again there is important variation. Some academies show significant positive coefficients, and 
some are significantly negative. Results for a single year are not conclusive. But compiling this 
information year after year, and including other outcomes in addition to test scores — such as 
attendance, credits earned, grades, behavior, and promotion to the next grade level or 
graduation from grade 12 — can be very helpful in guiding the continuous improvement of 
academies. 


PAGE 28 




COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Table 1: Regression coefficients predicting 2010 test scores 
Number of academies with significant and non-significant coefficients 


Test and grade level 

Number of academies where coefficient is 

Positive 

Not significant 

Negative 

English Language Arts, grade 10 

33 

331 

17 

English Language Arts, grade 11 

9 

274 

22 

Algebra 1, grade 10 

21 

306 

15 

Algebra 1, grade 11 

12 

152 

5 

Geometry, grade 10 

25 

324 

27 

Geometry, grade 11 

11 

269 

8 

Algebra II, grade 10 

12 

332 

17 

Algebra II, grade 11 

18 

268 

14 

Summative math, grade 11 

5 

261 

10 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


INFORMATION TO GUIDE IMPROVEMENT OF INDIVIDUAL 
ACADEMIES 

Academies vary a great deal in who enrolls, who stays, and whether performance improves. As 
the number of academies and Linked Learning pathways expands, it becomes increasingly 
important to monitor each program on an ongoing basis. Figure 37 shows how the information 
in this report can be summarized in an accessible way for an individual academy. Districts and 
schools can use this kind of profile to help guide continuous improvement. 
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COMPARING STUDENTS IN EACH ACADEMY WITH NON-ACADEMY STUDENTS AT THE SAME HIGH SCHOOL 


Figure 37: Sample profile for an individual pathway, 2009-10 


Academy of Construction, Manufacturing, and Engineering (ACME), Mount Diablo High School 

Key: compared to non-academy students at same school and grade level, 

^ indicates pathway had significantly more 
indicates pathway had significantly less 

Blank indicates no significant difference between pathway and non-academy students. 
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SUMMARY TABLES 

Table A-l: Comparing student characteristics in grade 10, all academies combined — 


Each cell shows percent of academy or non-academy students in each category 


Student characteristic 

Academy 

students 

(n=19,051) 

Non-academy students 

In all high 
schools 
(n=480,708) 

In high schools 
with academies 
(n=119,774) 

Female 

51.3 

48.5 

48.4 

Parent education 




Not a high school graduate 

23.2 

17.4 

21.5 

High school graduate only 

22.7 

19.8 

22.1 

Some college or AA degree 

17.0 

19.2 

17.0 

Bachelor's degree only 

11.6 

16.0 

12.4 

Post-graduate 

4.8 

9.5 

6.4 

Decline to state 

20.8 

18.1 

20.6 

Subsidized lunch eligible 

62.9 

48.5 

58.4 

Ever classified ELL 

57.4 

42.8 

53.5 

Initially FEP 

8.4 

8.0 

8.2 

Reclassified FEP 

32.9 

19.2 

24.9 

English learner now 

16.2 

15.7 

20.5 

Special education 

6.0 

9.6 

10.1 

Race or ethnicity 




American Indian 

0.5 

0.7 

0.5 

Asian 

8.9 

8.6 

9.2 

Pacific Islander 

0.8 

0.6 

0.7 

Filipino 

3.6 

2.7 

3.1 

Latino 

58.9 

48.7 

57.4 

African American 

8.6 

7.1 

7.9 

White 

16.2 

28.7 

18.7 

Two or more 

2.4 

2.9 

2.5 

Grade 9 ELA test scores 




Bottom quartile 

24.6 

25.6 

29.7 

Without special ed. 

22.0 

21.4 

25.0 

Second quartile 

34.6 

26.0 

27.8 

Without special ed. 

35.3 

26.6 

28.9 

Third quartile 

25.3 

24.1 

23.0 

Without special ed. 

26.4 

25.7 

24.8 

Top quartile 

15.5 

24.4 

19.5 

Without special ed. 

16.3 

26.3 

21.3 
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Table A-2: Comparing characteristics of academy and non-academy students in grade 10, one 
academy at a time — 


Each cell shows number of academies with or without significant differences (total n=400) 


Student characteristic 

Academy had 
more 
or higher 

No difference 
or mixed 

Academy had 
less 

or lower 

Too few 
students 
or no 

variance 

Female 

106 

220 

71 

3 

Parent education 

20 

339 

24 

17 

Subsidized lunch eligible 

43 

311 

35 

11 

Ever classified ELL 

55 

302 

37 

6 

Special education 

10 

258 

70 

62 

Race or ethnicity 





American Indian 

0 

1 

0 

399 

Asian 

13 

110 

25 

252 

Pacific Islander 

0 

3 

0 

397 

Filipino 

10 

42 

3 

345 

Latino 

53 

301 

41 

5 

African American 

8 

159 

18 

215 

White 

37 

171 

31 

161 

Two or more 

1 

42 

0 

357 

Grade 9 test scores 





ELA performance level 

114 

258 

28 

0 

Without special ed. 

97 

261 

36 

6 

ELA scale score 

126 

218 

56 

0 

Without special ed. 

108 

218 

68 

6 

Math performance level 

74 

287 

39 

0 

Without special ed. 

60 

291 

43 

6 

Math scale score 

98 

236 

66 

0 

Without special ed. 

78 

246 

70 

6 
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Table A-3: Comparing student characteristics in grade 11, all academies combined — 


Each cell shows percent of academy or non-academy students in each category 


Student characteristic 

Academy 

students 

(n=14,292) 

Non-academy students 

In all high 
schools 
(n=463,768) 

In high schools 
with academies 
(n=lll,648) 

Female 

53.2 

48.8 

48.9 

Parent education 




Not a high school graduate 

23.0 

17.2 

20.8 

High school graduate only 

22.6 

19.3 

21.4 

Some college or AA degree 

17.1 

19.5 

17.1 

Bachelor's degree only 

12.4 

16.5 

13.7 

Post-graduate 

4.8 

9.9 

6.9 

Decline to state 

20.1 

17.7 

20.1 

Subsidized lunch eligible 

62.0 

46.4 

55.7 

Ever classified ELL 

57.6 

42.4 

52.7 

Initially FEP 

8.4 

8.4 

8.4 

Reclassified FEP 

35.8 

19.7 

26.2 

English learner now 

13.4 

14.3 

18.1 

Special education 

6.0 

9.6 

9.5 

Race or ethnicity 




American Indian 

0.5 

0.7 

0.5 

Asian 

9.6 

9.0 

10.2 

Pacific Islander 

0.8 

0.6 

0.8 

Filipino 

4.0 

2.8 

3.4 

Latino 

58.0 

46.7 

54.0 

African American 

8.0 

7.0 

8.0 

White 

16.7 

29.7 

20.0 

Two or more 

2.5 

3.5 

3.2 

Grade 9 ELA test scores 




Bottom quartile 

21.4 

24.0 

26.1 

Without special ed. 

18.7 

20.0 

21.5 

Second quartile 

32.8 

25.5 

26.9 

Without special ed. 

33.4 

26.0 

27.8 

Third quartile 

28.4 

25.1 

24.8 

Without special ed. 

29.8 

26.7 

26.7 

Top quartile 

17.3 

25.4 

22.2 

Without special ed. 

18.2 

27.3 

24.0 
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Table A-4: Comparing characteristics of academy and non-academy students in grade 11, one 
academy at a time — 


Each cell shows number of academies with or without significant differences (total n=331) 


Student characteristic 

Academy had 
more 
or higher 

No difference 
or mixed 

Academy had 
less 

or lower 

Too few 
students 

or no 

variance 

Female 

91 

189 

48 

3 

Parent education 

18 

268 

24 

21 

Subsidized lunch eligible 

39 

265 

20 

7 

Ever classified ELL 

47 

254 

23 

7 

Special education 

8 

198 

47 

78 

Race or ethnicity 





American Indian 

0 

1 

0 

330 

Asian 

18 

84 

15 

214 

Pacific Islander 

0 

8 

0 

323 

Filipino 

14 

36 

2 

279 

Latino 

46 

262 

18 

5 

African American 

5 

107 

21 

198 

White 

16 

161 

25 

129 

Two or more 

0 

35 

2 

294 

Grade 9 test scores 





ELA performance level 

86 

202 

29 

14 

Without special ed. 

58 

231 

27 

15 

ELA scale score 

97 

173 

47 

14 

Without special ed. 

78 

175 

63 

15 

Math performance level 

46 

239 

32 

14 

Without special ed. 

36 

244 

36 

15 

Math scale score 

62 

200 

55 

14 

Without special ed. 

51 

197 

68 

15 
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